Tennessee Board of Regents: Developmental Studies Redesign Initiative

Improving Learning and Reducing Costs
A Summary of Program Outcomes

By Carol A. Twigg

From November 2006 to November 2009, the National Center for Academic Transformation (NCAT) conducted a program in course redesign in partnership with the Tennessee Board of Regents (TBR) called the TBR Developmental Studies Redesign Initiative. The goals of this partnership were to 1) build on the successful models and lessons learned from NCAT’s national programs to create a course redesign program focused on remedial and developmental mathematics and English within TBR institutions; 2) adopt new ways to improve student learning outcomes and demonstrate these improvements through rigorous assessment; 3) reduce instructional costs and free up instructional resources for other purposes; 4) produce a number of successful TBR course redesigns that can be spread to institutions throughout the system; and, 5) develop the internal capacity of TBR faculty and staff to continue the course redesign process. All of those goals have been achieved.

The TBR Initiative awarded grants of $40,000 to six institutions through a competitive process. NCAT engaged the TBR institutions in an extensive planning process to teach applicants its principles of course redesign. The NCAT process required faculty teams to follow a highly structured, iterative course redesign development process. With support from the Fund for the Improvement of Postsecondary Education (FIPSE), the Initiative was able to award a total of $240,000 in grants to participating institutions to support their redesign efforts. Twenty-seven proposals were submitted in July 2007, and the following six projects were ultimately funded : 1) Austin Peay State University: Elementary Algebra and Intermediate Algebra; 2) Chattanooga State Technical Community College: Basic Math, Elementary Algebra and Intermediate Algebra; 3) Cleveland State Community College: Basic Math, Elementary Algebra and Intermediate Algebra; 4) Columbia State Community College: Basic Reading, Developmental Reading, Basic Writing and Developmental Writing; 5) Jackson State Community College: Basic Math, Elementary Algebra and Intermediate Algebra; and, 6) Northeast State Technical Community College: Basic and Developmental Reading, Four of the six projects completed the redesign process and fully implemented their redesigns in spring 2009.

The results achieved by the TBR Initiative were very strong. The four successful redesigns impacted approximately 4,380 students. Three of the four demonstrated improved student learning as measured by direct comparisons of content mastery. (Because the fourth (Austin Peay) redesign eliminated developmental math courses completely, the team did not have comparable data on student learning outcomes.) .All four improved course completion rates (as measured by a final grade of C or better) All four reduced their instructional costs—on average by 36%. The annual savings for the four projects was $376,438. All four of the redesigns will definitely be sustained after the grant period is over.

NCAT’s recommendations to the TBR for future action included 1) the TBR should publicize the results of the Initiative among its 19 campuses; 2) the TBR should build on what was learned in the Initiative to revise its policy on remedial and developmental education; 3) the TBR should conduct a second round of the redesign program in developmental reading and writing; and, 4) the TBR should conduct further studies on issues that emerged from the Initiative.

Project Outcomes

What follows is a summary of the TBR course redesign projects’ final reports, which cover the following projects.

• Austin Peay State University Math
• Chattanooga State Technical Community College Math
• Cleveland State Community College Math
• Columbia State Community College Reading
• Columbia State Community College Writing
• Jackson State Community College Math
• Northeast State Technical Community College Reading

1) Did student learning improve (as measured by direct comparisons of content mastery)?

2) Did course completion rates improve (as measured by comparisons of final grades)?

3) Were instructional costs reduced?

4) Will the redesign be sustained after the grant period is over?

1) Did student learning improve (as measured by direct comparisons of content mastery)?

Yes

1. Cleveland State (compared common exam items)

• Student performance significantly improved in Basic Math, Elementary Algebra and Intermediate Algebra as measured by performance on common exam items.

2. Jackson State Community College (compared mean module post-test scores)

• Redesign students increased their average post-test scores in all courses by 15 points.

• In spring 2008, 11 traditional sections (220 students) and 13 redesigned sections (356 students) were offered on JSCC’s main campus. In the traditional sections, the average post-test score for all 12 modules was 73%. For the redesigned sections, it was 82%.

• In fall 2008, the average increased to 85%. In spring 2009 the average increased to 88%.

3. Northeast State Technical Community College (compared common final exams and common pre/post-tests)

• Students in the redesigned course performed better on a common final exam. In the traditional course, the final exam scores averaged 81% while in the redesigned course, final exam scores averaged 86% in fall 2008 when the redesign was fully implemented.

• Analysis of pre-test and post-test Nelson Denny scores revealed that students in the redesigned course obtained a greater gain in their learning and reading skills. In the traditional course, the average gain was 11 points, while the average gain was 15 points in the spring 2008 pilot and 21 points in the fall 2008 full implementation. Students in the redesigned course made more progress in improving their reading skills than those in the traditional reading course. To compare learning in the redesigned course with the learning in the traditional course, Northeast examined students’ scores on psychology learning outcomes as coded in assigned student research papers.

No Difference

1. Columbia State Community College Reading (compared pre/post-test scores)

• Analysis of pre-test and post-test Nelson Denny scores revealed no significant difference in the gain of learning and reading skills. In the traditional course, the average gain was 1.1 points, while the average gain in the redesigned sections was 1.6 points in fall 2008 and 1.1 points in spring 2008.

Unknown (No Data Reported)

1. Austin Peay State University

• Because Austin Peay’s redesign eliminated developmental math courses completely, the team did not have comparable data on student learning outcomes for the traditional and redesigned courses.

2. Chattanooga State Technical Community College

• Chattanooga State did not collect learning outcome data. They reported only final grades.

3. Columbia State Community College Writing

• Columbia State did not collect learning outcome data. They reported that they could not agree on how to score writing samples.

2) Did course completion rates improve (as measured by comparing final grades)?

Yes

1. Austin Peay State University (compared calculated completion rates in subsequent college-level courses)

• Prior to the redesign, students were required to complete one or more developmental courses before enrolling in a core mathematics course. The success rates for the developmental courses were 53% for Elementary Algebra and 51% for Intermediate Algebra.

• Austin Peay compared how well developmental math students performed in two subsequent college-level courses (Mathematical Thought and Practice and Elementary Statistics) both before and after the redesign. To do this, the team calculated a success rate for what percentage of the entire developmental math population was successful in the two core courses, taking into account that a significant portion of them never enrolled in a college-level math course.

• This calculation showed that the percentage of students who succeeded (grade of D or better and completed the requirements for removing deficiencies using the SLA format) in the redesigned mathematics courses was significantly higher than the success rates that occurred when students were required to complete the developmental mathematics requirement before enrolling in the college-level courses.

• Prior to the redesign, 33% of developmental students who enrolled in Mathematical Thought and Practice successfully completed the course. After the redesign, that rate averaged 71%.

• Prior to the redesign, 23% of developmental students who enrolled in Elements of Statistics successfully completed the course. After the redesign, that rate averaged 54%.

2. Cleveland State Community College

• The completion rate for Elementary Algebra increased from 50% before to 68% after the redesign.

• The completion rate for Intermediate Algebra increased from 57% rate before to 74% after redesign.

• Students in Basic Math succeeded at a higher rate of 65% during the 2008 fall semester, but returned to the historical average of 50% during the 2009 spring semester. Basic Math still suffers from high failure rates and the problems in the course will be addressed in an attempt to improve student success in the future.

• The number of students who passed a developmental math course increased by 29%. In previous years, 657 of 1,234 students (54%) passed at least one developmental course compared with 845 of 1,201 students (70%) in AY 2008-2009.

• The number of students exiting the developmental math program increased by 32%. In previous years, 330 of 595 students (55%) exited the program compared with 436 of 591 students (74%) in AY 2008-2009. Overall, though, the number of students exiting developmental math and completing a college level math course increased significantly.

3. Jackson State Community College

• Because JSCC radically changed the structure of its developmental math program, it is impossible to show comparative completion rates of individual courses other than in the spring 2008 pilot semester when all three developmental courses were offered using the module concept. Only the pedagogy differed.

• In spring 2008, 11 traditional sections (220 students) and 13 redesigned sections (356 students) were offered. In the traditional sections, 41% of the students received a passing grade (C or better) compared to 54% of students in the redesigned sections. The spring 2008 pass rate was comparable to JSCC’s historical 42% pass rate.
• In Basic Math, 47% of traditional students received a passing grade compared with 54% of redesign students.
• In Elementary Algebra, 32% of traditional students received a passing grade compared with 66% of redesign students.
• In Intermediate Algebra, 48% of traditional students received a passing grade compared with 44% of redesign students.

• From fall 2008 onward, students in the redesign proceeded through the required modules at their own pace with the expectation that they would complete four modules during each term or the remaining number needed. When one semester ended and another began, students simply resumed work on the modules not completed. Thus, both the elements of “course” and “time” were removed. Since students “completed” “courses” under very different conditions, it is only possible to compare overall completion rates in the traditional and redesigned format for the developmental math program as a whole. In the redesign, students enrolled in a “shell course.” The grade awarded was the average of the four (or fewer if required) modules completed.

• In fall 2008, 57% of the 711 students enrolled in redesigned sections received a passing grade (C or better) compared to 41% in the spring 2008 traditional sections.

• In spring 2009, 59% of 670 students enrolled in redesigned sections received a passing grade.

• Thus, JSCC increased the overall student success rate in developmental math by 44%.

4. Northeast State Technical Community College

• The student success rate (a grade of C or higher) in the traditional course was 58% while in the redesigned course, the success rates were 58%, 67%, and 54% in fall 2007, spring 2008 and fall 2008 respectively, an overall average of 60%.

• Grade distributions of the redesigned course showed that significantly more students had As and Bs than in the traditional course.

• The percentage of students who passed the redesigned DVMT101 was substantially higher (51%) than in the fall 2007 traditional format (33%).
  
• The team is concerned, however, that the percentage of students who passed both courses in one semester has dropped from 49% to 33%.

No

1. Chattanooga State Technical Community College

• The number of students achieving a C or better in Elementary Algebra decreased from 42% to 29% using the most successful redesign semester as a comparison.

• The number of students achieving a C or better in Intermediate Algebra decreased from 42% to 24% using the most successful redesign semester as a comparison.

2. Columbia State Community College Writing

• The average number of students earning grades of C or better increased from 59% in the traditional format to 66% in the redesigned format in one term and decreased to 52% in a subsequent term.

3. Columbia State Community College Reading

• Using a weighted average of all pilot terms, the average number of students earning grades of C or better decreased from 57% in the traditional format to 39.3% in the redesigned format.

3) Were instructional costs reduced?

Yes

1. Austin Peay State University

• APSU reduced the cost of providing developmental math by 52% (a savings of $209,248.) The university realized the savings by:

1. Eliminating 52 developmental math sections each year, which previously cost $402,804 ($384,500 for the instructors and $18,304 for the tutors who supported the courses.)
2. Adding 32 enhanced sections of the two core courses, six of Mathematical Thought and Practice and 10 of Elements of Statistics. (The cost of the redesign includes only half of these enhanced sections since half of these sections would have been offered in the traditional format for the 50% of developmental students who previously enrolled in these two courses.) The cost of six enhanced MT&P sections and 10 enhanced Statistics sections was $121,956.
3. Adding 32 student-led SLA workshops required as a co-requisite with a core mathematics course, which cost $71,600.

The total cost of the redesign was $193,556.

• In addition, reducing the number of sections offered saved classroom space. Estimates indicate that the university saved 70 classroom hours per week with the redesign.

2. Cleveland State Community College

• Significant cost savings have been realized as a result of the redesign project. In the traditional model, Cleveland State offered 55 24-student sections in fall and spring, 45 of which were taught by full-time faculty at a cost of $256,275 and 10 by adjuncts at a cost of $14,400. The total cost of the traditional course was $270,625.

• In the redesigned model, Cleveland State offered 77 18-student sections in fall and spring, all of which were taught by FT faculty at a cost of $219,258. The total cost savings was $51,367 or 19%. This is a significant cost savings at a small college like Cleveland State with a math department of eight faculty members and one staff member.

• Faculty productivity has risen by 23%. The average student load per faculty member went from 106 to 130, and the FTE teaching load per faculty member went from 21.2 to 26.0. Faculty used to teach five sections per semester. In the redesigned environment, faculty members will teach 10-11 sections, which means that they will handle 150+ students each semester, and work 8–10 hours in the lab.

• The personnel mix of the department has also shifted. Increased faculty productivity has enabled the department to eliminate the use of adjunct instructors while increasing course offerings. Overloads have also been reduced as a result of the redesign project. One faculty member retired and was replaced with a staff member. Also, a full-time lab person in Cleveland left and was replaced with part-time math lab tutors. These shifts in personnel not only saved money but also proved to work better in staffing the lab and serving student needs. Other savings include less copying due to online homework and testing.

3. Jackson State Community College

• In the traditional model, JSCC offered 89 sections of 20 – 24 students during fall 2006 and spring 2007, 63 of which were taught by full-time faculty at a cost of $290,871 and 26 by adjuncts at a cost of $37,778. The cost of tutors was $4,510, bringing the total cost of the traditional course to $333,159.

• In the redesigned model, JSCC offered 71 sections during fall 2008 and spring 2009; 44 sections enrolled 30 students and 27 enrolled 24 students. The number taught by full-time faculty was 37 at a cost of $170,829, and the number taught by adjuncts was 34 at a cost of $49,402. The cost of tutors was $38,298, bringing the total cost of the redesigned course to $258,529.

• The cost-per-student was reduced from $177 to $141, a 20% decrease.

The savings described above reflect what happened in the first year of redesign. JSCC anticipates even greater savings in the future due to three factors:

• JSCC believes that the ratio of tutors working in the SMART Center to the number of students enrolled may decline in the future since more tutors were needed in the initial redesign development process than will be needed now that the program is fully implemented.

• By changing the requirements for developmental math completion, JSCC theoretically reduced the number of sections/modules they needed to offer by 31%. (During the 2008-09 academic year, 1836 students were enrolled in developmental math. JSCC needed to offer the equivalent of 15,241 modules to serve these students under the new policy. Assuming similar placement distributions, JSCC would have had to offer 22,032 modules under the old policy.) This reduction was not fully realized in the first year of implementation.
  
• Approximately 18% more students are exiting the developmental math program sooner in the redesigned format than previously, which will reduce still further the number of sections needed. This 18% translates to seven sections which do not have to be offered.

4. Northeast State Technical Community College

• Northeast reduced costs by 51%. The traditional reading course was taught in 24 small sections (~17 students) annually. Half were taught by full-time faculty and half by adjuncts. Small sections entailed a high delivery cost; the total cost of offering the traditional course was $80,832.

• The redesigned course included one section each term of 275 (fall) and 137 (spring) students each. Each section was team-taught by two full-time faculty members, and no adjuncts were used. The team added trained reading professionals to work with students in the Reading Center at $15 per hour. The total cost of the redesigned course was $39,639, which represents a savings of $41,119, a 51% reduction.

Unknown (No Data Reported)

1. Chattanooga State Technical Community College

2. Columbia State Community College Reading

3. Columbia State Community College Writing

4) Will the redesign be sustained after the grant period is over?

Yes

1. Austin Peay State University

• The TBR Strategic Plan placed developmental studies in the community college; the closest TBR community colleges are in Nashville, more than 50 miles from Clarksville, where APSU is located. Retaining a way to address mathematics deficiencies at the university level provides higher education access for students who are time and place bound and most likely would not be able to attend the nearest community college. At this point, the university has no plans to change the way mathematics deficiencies are addressed.

• Student success rates have improved as has the quality of instruction. The fall-to-fall retention rate of students with mathematics deficiencies is higher for students enrolling in enhanced core courses than it was for students who were required to enroll in developmental mathematics courses. Greater consistency has been achieved in multiple sections of courses. Common questions embedded in exams for both the enhanced and regular sections of the two core courses suggest that the same rigor is being maintained in both formats.

• In response to the redesign successes, the university applied for and received a $2 million Title III Grant, which will provide funding over the next five years to support course redesign. The focus of the redesign effort is on core courses which have D-F-W rates of 30 percent or more.

2. Cleveland State Community College

• The project has been a resounding success. Due to the success of the project, the math department has continued to expand course redesign to include three college-level courses: College Algebra, Finite Math and Introductory Statistics. These redesigns have also achieved impressive results.

• Cleveland State also plans to redesign three additional courses and to implement those courses in fall 2009: Basic Calculus, Precalculus I and Precalculus II. Once these courses are offered under the new format, approximately 95% of the students at Cleveland State will be taking math courses in the redesigned format.

• John Squires, the project leader, recognizes the advantages that can be realized when you look at department redesign as compared to course redesign. By including both college-level math courses and developmental math courses in the redesign, the department has been able to increase efficiency and reduce costs. Economies of scale are more easily realized the bigger the redesign project is. Also, redefining faculty roles and duties is easier to do when the entire department and most of its course offerings are involved in the redesign.

3. Jackson State Community College

• The sustainability of this project is not in question. During the 2008-2009 academic year, all sections of developmental mathematics followed the redesign format.

• Administration has supported the redesign from the beginning and continues to support it. The entire campus has embraced SMART Math and marvels at the success rates. The innovations pioneered in this project are being used in college-level mathematics courses and others that are being designed.

• A number of benefits to students will continue to accrue, including 1) greater flexibility of students to schedule coursework, 2) the elimination of time spent on unnecessary coursework, 3) the opportunity to move more quickly toward the students’ chosen career goals, and 4) a greater chance for success. All of these benefits make it possible for the students to save money as they are allowed to register for fewer hours in developmental math than would have been required in the past.

4. Northeast State Technical Community College

• The team believes that the redesign of the reading course is sustainable due to two major reasons. First, the redesigned course was implemented to include all reading students in fall 2008, and no major problems were encountered. Second, the redesigned course generates significant savings for the college; therefore, funding will not be an issue to sustain the redesign.

Unknown

1. Chattanooga State Technical Community College

2. Columbia State Community College Reading

3. Columbia State Community College Writing

Final reports from each completed redesign are at http://www.thencat.org/States/TN/TN%20Project%20Descriptions.htm. Final reports include learning outcome data, course completion data, cost reduction data, a discussion of the most important pedagogical techniques that led to increased learning, a discussion of the most important cost reduction techniques that led to reduced costs, a discussion of implementation issues encountered during the redesign process, and a discussion of future sustainability of the redesign.